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The aim of the current study was to investigate the effects of the modified and enhanced 
Aussie Optimism Positive Thinking Skills Program (AO-PTS) on Year 4 and 5 children's 
social and emotional learning (SEL) skills. AO-PTS is a universal-school based program that 
is implemented by class teachers as part of regular school curricula and was developed 
for the prevention of depression and anxiety. The study comprised a total of 683 Year 
4 and 5 students from 10 private primary schools in Western Australia. Students were 
assessed on two subscales of emotional attribution at school whilst parents reported on 
their children's externalizing and internalizing problems outside of school and at home. Two 
analyses were conducted: seven intervention schools were assessed at pre- and post-test 
(Analysis 1) and pre-post change in three intervention schools were compared to pre-post 
change in three matched control schools (Analysis 2). Results from Analysis 1 showed 
that the intervention children had increased in their overall emotional attribution accuracy 
and decreased in total difficulties and hyperactivity; Results from Analysis 2 revealed 
no intervention effect on emotional attribution accuracy or internalizing or externalizing 
problems. These findings suggest that the enhanced AO-PTS's effects on SEL were not 
evident in the short-term period after intervention. The non-significant findings and future 
directions for AO-PTS research and program modification were discussed. 
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INTRODUCTION 

Well-developed social and emotional skills in childhood provide 
resilience for both the home and school environment and in later 
life (Zins and Elias, 2006; Payton et al, 2008). An ability to negoti- 
ate social relationships and have good awareness of one's own and 
others' emotions provides a backdrop onto which children can 
either thrive in and outside of school or alternatively that they can 
be faced with a host of potential problems (Bernard et al., 2007; 
Denham and Brown, 2010). Well-developed social and emotional 
skills are related to good academic outcomes; conversely, children 
with poorly developed emotional skills may be more likely to have 
behavioral adjustment problems, more sexual partners, and are 
more likely to use hard drugs (Denham and Brown, 2010; Hessler 
and Katz, 2010). Barblett and Maloney (2010) highlighted that the 
"empirical research is overwhelming in providing evidence that 
strong growth in social and emotional competence and wellbeing 
underlies all later growth and development" (p. 14). 

Social and emotional skills are important for all children, not 
just those at-risk. A recent report on the social and emotional 
health of Australian children reported a steady decline in students' 
ratings of their social and emotional well-being that began in the 
middle years of primary school (Bernard et al, 2007). In addition 
the ratings by students of their social and emotional well-being 
did not differ between those in the highest and those in the low- 
est SES ranges. This indicates that social and emotional health and 



the skills that promote these are relevant to children from all walks 
of life. 

Social and emotional learning (SEL) is the process of acquir- 
ing the knowledge, attitudes and skills that permit children to 
identify and cope with their emotions, set and work toward goals, 
be able to empathize with others, make decisions and be able to 
effectively navigate interpersonal relationships, including keeping 
and maintaining friendships (Payton et al., 2008). SEL capabilities 
are inextricably linked with a child's social and emotional well- 
being (AIHW, 2012). Denham and Brown (2010) outline five 
core SEL competencies: (1) self-awareness; (2) self-management; 
(3) social awareness; (4) responsible decision making; and (5) 
relationship/social skills. These essential skills allow children to 
regulate their emotions when they get angry or distressed, start 
up and continue relationships with their peers and other people, 
and deal with challenging situations when they arise. 

A considerable amount of literature has been published on 
SEL universal programs and point to a range of positive bene- 
fits for children. These benefits are demonstrated by the findings 
from two recent meta-analyses of universal school-based SEL pro- 
grams. One meta-analysis was conducted on 213 school-based 
universal SEL programs that included 270, 034 kindergarten 
through to high school students (Durlak et al., 201 1) and the sec- 
ond meta-analysis was conducted on 180 studies (including 277, 
977 kindergarten through to Year 8 students; Payton et al, 2008). 
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These two meta-analyses indicate that SEL programs enhanced 
students' behavioral adjustment in the form of increased proso- 
cial behaviors (more positive social behaviors, prosocial atti- 
tudes toward aggression, more positive attitudes toward self and 
others); reduced conduct problems; reduced internalizing prob- 
lems (lower levels of emotional distress in the form of anxiety 
and depressive symptoms), and improved academic performance 
(with an 11 point percentile increase in academic achievement 
test scores) (Payton et al., 2008; Durlak et al, 2011). Effect sizes 
of the universal SEL programs ranged from 0.22 to 0.57 (Durlak 
et al., 2011). In addition, the findings indicated that school teach- 
ers and other school staff can effectively conduct SEL programs in 
schools; in fact the effect sizes for outcomes relating to attitudes, 
positive social behavior, conduct problems, emotional distress, 
and academic performance were better when teachers ran the pro- 
grams, compared to when non-school personnel conducted the 
programs (Durlak et al., 201 1). The authors of both reviews con- 
cluded that universal school-based SEL programs can be effective 
and have a positive impact on multiple areas of students' lives. 

More recently, many universal school-based SEL studies have 
been conducted to investigate the impact of SEL programs on 
children. For instance, one large scale randomized clinical trial 
evaluated an adaption of the Promoting Alternative Thinking 
Strategies curriculum (PATHS) on pre-school age children. After 
receiving the PATHS program, the students' in the 10 inter- 
vention classrooms were rated by their parents and teachers as 
more socially competent, less socially withdrawn and had higher 
emotion knowledge skills, compared to students in the control 
classrooms (Domitrovich et al, 2007). Another PATHS study, 
which was a longitudinal evaluation of children followed from 
Grade 1 to Grade 3, found a reduction in teacher and peer 
reported aggression and an increase in prosocial behavior, as 
well as teacher-reported improvements in students' academic 
engagement (Conduct Problems Prevention Research Group, 
2010). A study of Head Start — an emotion-based prevention pro- 
gram in 2- to 5-year old rural and urban children — reported 
greater increases in emotion knowledge, emotion regulation, pos- 
itive emotion expression, and social competence (Izard et al., 
2008). Further evidence comes from studies on children in 
middle primary school. For instance, the "Roots of Empathy" 
program was implemented on 4th- to 7th-grade children and 
reported improvements in prosocial behavior, proactive and reac- 
tive aggression, however, empathy and perspective taking did not 
show improvements (Schonert-Reichl et al., 2012). Collectively, 
these findings indicate that universal school-based SEL programs 
are promising for improving children's social and emotional com- 
petence. However, little research has been conducted on children 
in middle primary school and it is not known if depression 
and anxiety prevention universal school-based programs can also 
positively impact on children's SEL skills. 

Another universal school-based program is the Aussie 
Optimism Program: Positive Thinking Skills (AO-PTS). Although 
AO-PTS was originally developed as a program to prevent depres- 
sion and anxiety in children in middle childhood, the contents 
of the program also fit well with core aspects of SEL programs. 
Many SEL programs are long in duration (e.g., 25 lessons and 
three years, respectively, Conduct Problems Prevention Research 



Group, 2010; Schonert-Reichl et al., 2012) or target specific 
behaviors such as aggression (Conduct Problems Prevention 
Research Group, 2010). In contrast, AO-PTS is delivered by 
school teachers as part of the regular curriculum over 10 weekly 
sessions. Furthermore, the contents of AO-PTS are consistent 
with the five core interrelated factors, as outlined by Collaborative 
for Academic, Social, and Emotional Learning (Collaborative for 
Academic Social and Emotional Learning, 2005), that SEL pro- 
grams need to address, including: accurately assessing one's feel- 
ings, regulating emotions and working toward goals, being able to 
empathize with others, establishing and maintaining healthy and 
rewarding relationships and making responsible decisions. 

Research conducted on Aussie Optimism Programs (AOP) 
aimed at older children indicate that AOP may have some impact 
on children's SEL outcomes, however, the findings have been 
mixed. For instance, these studies have reported reduced par- 
ent reported internalizing symptoms in Grade 7 students from 
disadvantaged schools, but not for externalizing or social skills 
(Roberts et al., 2010); reduced total difficulties and conduct prob- 
lems in Grade 8 students (Swannell et al., 2009); and reduced 
likelihood of using tobacco and alcohol at the end of Grade 8 
when the children received AOP in Grade 6 and 7, but only 
when the teachers implementing the program received coaching 
(Roberts et al., 201 1). These findings have been somewhat incon- 
sistent and tell us little about AOP impact on SEL for younger 
children. 

To date, five studies have been conducted on the AO-PTS. 
A pilot study conducted on 72 8- to 9-year old children from 
four low socioeconomic (SES) state primary schools, who were 
randomly allocated to either intervention or health education 
as usual, found lower levels of depression and more positive 
attribution styles in intervention children, compared to control 
children (Rooney et al, 2006). In addition, less of the interven- 
tion children maintained a diagnosis of depression at post-test 
or developed a depressive disorder at 9-month follow-up, com- 
pared to control children (Rooney et al., 2006). Similarly, a large 
scale randomized trial of AO-PTS, conducted on 910 Year 4 
and 5 children from 22 low SES schools, also found that inter- 
vention children were significantly less depressed at post-test 
as well as less parent-reported total difficulties at post-test and 
6 months following the intervention when compared to control 
children (Rooney et al, 2013). Both intervention and control 
children increased in parent-reported prosocial behaviors at post- 
test, 6- and 18-month follow-up assessments. Conversely, no 
significant differences were found between the children who had 
received AO-PTS and those who received regular health edu- 
cation on outcomes relating to hyperactivity, conduct or peer 
problems (Rooney et al, 2013). A 30-month follow-up study 
was conducted with the same cohort of students to examine the 
medium-term effect of AO-PTS. Results showed that there was a 
significant reduction in hyperactive behaviors for the intervention 
children; however, no significant differences were found in the 
depressive, anxiety and attribution styles across groups (Morrison 
et al., 2013). Furthermore, two follow-up studies conducted at 
42- and 54-months post intervention, aimed at determining the 
longitudinal effect of the AO-PTS, found no significant reduc- 
tion in depression, anxiety and attribution styles across groups 
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(Johnstone et al., 2014). The AO-PTS has now been enhanced to 
include more content on social and emotional competence and 
a less complex cognitive component which is more developmen- 
tally appropriate (Rooney et al., 2009). A small pilot study with 
the enhanced version of the AO-PTS was conducted in a rural 
setting and found that control group children had significantly 
increased in the parent-reported levels of peer problems at post- 
test when compared to the intervention children (Lacey et al., 
submitted). 

In summary, research on universal school-based programs has 
found enhanced behavioral adjustment, with increased proso- 
cial behavior and academic performance, and reduced conduct 
and peer problems; however, many of these programs involve 
long implementation periods, target certain behaviors, and little 
is known about the effects of depression and anxiety preven- 
tion programs on children's SEL skills. Research on older children 
using AOP programs have generally been conducted over 20 ses- 
sions and have reported mixed findings related to SEL factors. 
Although there have been a number of studies conducted on AO- 
PTS, particularly with relation to its effect on depression and 
anxiety, little research to date has reported on the effects of the 
enhanced version of AO-PTS on children's social and emotional 
skills. 

To address these identified gaps in the literature, given that 
there is a decline in children's self-reported social and emotional 
well-being beginning in middle childhood, the present study 
aimed to investigate the impact of the enhanced AO-PTS program 
on the social and emotional development of Years 4 and 5 chil- 
dren. Private schools were chosen as there is a paucity of research 
on students in these schools and evidence suggests that aspects of 
these schools may present challenges to the mental health of chil- 
dren who attend them (Watt, 2003). In addition, children from 
both high and low SES do not differ in these subjective reports 
of well-being. The implications of positive findings would be that 
there is a universal school-based program that can be successfully 
implemented by teachers as part of the regular school curriculum, 
which not only has strong evidence for its preventative effects on 
depression, but also positively impacts on children's general SEL 
skills, with the potential to benefit many areas of their home and 
school life. To this end, the following hypotheses are proposed: 
At post-test, parents of students in the intervention group will 
report lower levels of internalizing and externalizing symptoms 
and higher levels of prosocial behaviors and lower levels of total 
difficulties in their children (as measured by the SDQ-P) than 
parents of students in the control group (Hypothesis 1). At post- 
test, the intervention group will report higher levels of emotional 
attribution accuracy (as measured by the ACES) than the control 
group (Hypothesis 2). 

METHODS 
PARTICIPANTS 

Participants were 683 Years 4 and 5 students from 10 private 
primary schools in Western Australia, which represents 56.63% 
of the available 1206 Year 4 and 5 in these schools. There were 
approximately equal numbers of males and females (350 boys 
[51.2%] and 333 girls [48.6%]). At pre-test, participants were 
aged between 8.75 and 11.58 years. The large age range reflects 



the fact that some children had their pre-test in 2010, some in 
201 1, and others in 2012. A breakdown of the children's ethnicity 
is provided in Table SI. 

The gender and number of students in each year group in both 
the intervention and control conditions are presented in Table S2. 
Students in the intervention group received the AO-PTS program 
delivered by their teachers as part of their normal school cur- 
riculum; students in the control condition received their regular 
health education lessons. 

This study used a sub-set of data from an ongoing larger AO- 
PTS research which will collect data from 30 private schools which 
have been randomly selected from a range of socioeconomic 
areas in Western Australia. SEL universal interventions have been 
found to be equally beneficial for both socioeconomically advan- 
taged and disadvantaged students (Trentacosta and Fine, 2009). 
To date, data for Year 4 and Year 5 students from seven interven- 
tion schools and three control schools were available and this is 
the data used in the current study. 

RESEARCH DESIGN 

For the overarching study, before pretesting, 30 private schools 
were randomly allocated to one of two conditions — school-based 
intervention or control — with 15 schools in each condition. This 
research study used a nested cohort design (Murray and Hannan, 
1990): schools were allocated to either the intervention or con- 
trol condition and the students were then followed as a cohort 
to determine the effects of the intervention. This research design 
consisted of two categorical random effects (school and student), 
one categorical fixed effect (group: intervention, control), and 
one ordinal fixed effect (time: pre, post). This design gener- 
ated a hierarchical data structure in which time is nested within 
student and student is nested within school. There are two fam- 
ilies of outcome variables: (i) self-reported emotional knowledge 
and recognition (measured by ACES), (ii) parent evaluations of 
the internalizing and externalizing problems which include the 
prosocial behavior and total difficulties, in their child (measured 
by SDQ-P). 

INSTRUMENTS 

The Assessment of Children's Emotional Skills (ACES; Schultz et al., 
submitted) was used to assess children's self-reported emotional 
knowledge and emotional recognition ACES includes three sec- 
tions that reflect the child's accuracy of emotion attribution 
relating to social behaviors, social situations, and facial expres- 
sions (Schultz et al). This study used the social behaviors and 
social situations subscales. Each of these sub-scales included 15 
one- to three- sentence items to which the child labeled the por- 
trayed character's feeling as "happy," "sad," "mad," "scared," or 
"no feeling." Children received a score of either 1 (correct) or 
0 (incorrect) for each item. Three items were related to each of 
the emotions under each subscale. Twelve of the 15 items on 
each subscale, excluding the ambiguous ("no feeling") items, were 
summed to produce a total score for each of the subscales yield- 
ing a score out of 12; the total scores of these the two subscales 
were summed to produce the ACES total score. The scores for 
emotional assessment range from 0 to 24, with high scores indi- 
cating greater emotional attribution accuracy. The reliability for 
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the three subscales (facial expression, social behavior, and situa- 
tions) is moderately high (r = 0.68; Schultz et al.). The internal 
consistency coefficient for the total emotion accuracy scale with 
the current sample at pre-test was moderate (a = 0.61); internal 
consistency coefficients for the social situation and social behav- 
ior subscales were both low (ct = 0.47) and ranged between 0.16 
and 0.43 for the emotion subscales that constituted these two 
subscales. Thus, all three of these scales, which included 10 or 
more items, did not obtain a Cronbach's alpha of 0.7+ indicating 
good internal consistency (Tabachnick and Fidell, 2007). In addi- 
tion, the subscales of the social behaviors and social situations, 
which contained less than 10 items each, did not meet a suffi- 
cient Cronbach's alpha of around 0.6 (Loewenthal, 2001). The 
test-retest reliability in the current study for the Total Emotion 
Attribution Score was moderate (r = 0.56) and for the Behavioral 
Situations and Social Situations were 0.42 and 0.52, respectively. 
In addition, the ACES has been found to correlate well with trait 
emotional intelligence (Mavroveli et al., 2009). 

The parent version of the Strengths and Difficulties 
Questionnaire (SDQ-P; Goodman, 1999) was used to assess 
children's overall psychological adjustment and provide a mea- 
sure of social and emotional competence (see Supplementary 
Material). The parent version of the SDQ-P measures inter- 
nalizing and externalizing problems in the home, designed 
for use by parents/caregivers of children 4-16 years. There are 
25 items measuring internalizing and externalizing problems 
across five subscales — Emotion Symptoms, Conduct Problems, 
Hyperactivity/Inattention, Peer Difficulties, and Prosocial 
Behavior. A Total Difficulties Score is computed by summing 
across the first four subscales. Parents rate each given item as 
either "not true," "somewhat true," or "certainly true." Internal 
consistency coefficients range between 0.76 and 0.82 for the total 
and summative scores. The internal consistency coefficient for 
the current sample at pre-test for the Total Difficulties Score 
was high (ot = 0.84) and ranged between 0.64 and 0.82 for the 
five subscales, indicating sufficient or good internal consistency 
(Loewenthal, 2001; Tabachnick and Fidell, 2007). The SDQ-P 
has strong 2-week test-retest reliability (0.96 for the total score; 
Goodman, 1999). The test-retest reliability in the current 
study was very high for the Total Difficulties Score (r = 0.92) 
and ranged between 0.73 and 0.82 on the five subscales. The 
instrument correlates well with the Rutter Parent Scale and the 
Child Behavior Checklist (Goodman and Scott, 1999). Numerous 
studies have found the SDQ to have good concurrent and 
discriminant validity in non-clinical samples (Goodman, 1999; 
Goodman and Scott, 1999; Muris et al, 2003). 

A Demographic questionnaire was used to collect background 
demographic information from parents regarding: socioeco- 
nomic status (SES), ethnic origin, family structure, family health, 
history of mental health problems, and contact details. 

PROCEDURE 

This study used a subset of available data from a larger on-going 
Aussie Optimism Positive Thinking Skills project. The procedure 
for the larger project involved the selection of thirty primary 
schools from all private schools in the Perth Metropolitan area 
who were assigned at random to either the intervention or control 



condition. School principals were invited to participate with the 
understanding that assessments would take place at both pre- and 
post-tests. The larger project researchers will be collecting 6- and 
18-month follow-up data, whereas the current study used the pre- 
and post-test data from seven intervention schools and three con- 
trol schools only. The teachers from participating intervention 
schools attended an 8 h training course in which they were taught 
the principles of the Positive Thinking Skills program and how to 
effectively run groups. 

The enhanced Aussie Optimism Program: Positive Thinking 
Skills Program (AO-PTS; Rooney et al, 2009) is a universal school- 
based program that involves 10 1-h weekly sessions which are 
delivered as part of the school curriculum. The 10 sessions tar- 
get areas such as the identification and awareness of comfortable 
and uncomfortable feelings, understanding feelings in others, 
cognitive re-structuring, a fear hierarchy, pleasant events schedul- 
ing and weekly relaxation training. In later modules, there is 
a focus on situations relevant to peer and family relationships. 
Each of the 10 modules has specific learning outcomes and teach- 
ing strategies, such as teacher-led group discussions, role plays, 
and skills practice sessions. There are three program booklets 
in the enhanced AO-PTS program: a teacher resource manual 
(Teacher Resource: Aussie Optimism Positive Thinking Skills; 
Rooney et al., 2009), a student workbook (Nesa et al., 2004), and 
a self-directed parent manual (Nesa and Rooney, 2004). Table S3 
outlines the topics and aims of each of the 10 modules in the 
teachers and student workbooks. 

The intervention schools and teachers were offered 5 x 1 h 
coaching sessions. The active consent of children and parents was 
sought after the study has been fully explained to them via school 
meetings and a formal information and consent form was sent 
home. The assessment battery was administered to the children 
by trained research assistants to provide baseline pre- and post- 
test measures of emotional competency. Measures for the parents 
were sent home through the school in sealed envelopes at pre- and 
post-test. The weekly intervention sessions were run in the usual 
class room groups. 

The groups were conducted by teachers at the school at con- 
venient designated times over 10 weekly 1-h sessions. Although 
monitoring of adherence to the assigned interventions was 
planned using implementation checklists, log books, reports, 
interviews, and observations by research assistants, resistance 
was received by teachers regarding filling-in the integrity check- 
lists and having their sessions observed. One possible reason for 
teachers' reluctance to complete the integrity check-lists is the 
potential extra workload. Integrity check-lists required the indi- 
vidual assessment of attendance and level of participation in 
each students' log-book. Hence, with the risk of many schools 
withdrawing from the study, these aspects of the project were 
dropped. 

DATA ANALYSIS 

The aim of the present paper was to determine whether there 
were intervention effects at the post-intervention assessment. 
These data were analyzed with a multi-level mixed effects linear 
regression model (Bryk and Raudenbush, 1987) as implemented 
through SPSS's Generalized Linear Mixed Models (GLMM: SPSS 
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Version 19) procedure. In order to test the relationships between 
the fixed effects and the outcomes, GLMM assumed a nor- 
mal probability distribution for each outcome and linked it 
to the fixed effects with an identity function. If the outcomes 
did not have a normal distribution, then the parameter esti- 
mates of the covariance matrix were computed with robust 
statistics. Finally, in order to optimize the likelihood of con- 
vergence, separate GLMM analyses were run for each of 13 
student-reported ACES outcomes (Total Emotion Attribution, 
Total Behavior and Social Situations Attribution, with related 
five feelings subscales) and for each of the parent-reported 
SDQ outcomes, the Total Difficulties (Emotional Symptoms, 
Conduct Problems, Hyperactivity /Inattention, Peer Relationship 
Problems) and Prosocial Behavior. As analysing each out- 
come independently of the others would inflate the fami- 
lywise error rate, the per-test alpha was corrected to con- 
trol the inflation. In order to conserve statistical power, the 
alpha correction within groups of conceptually related out- 
comes, rather than across the entire set of outcomes was 
applied. 

POWER 

Data from previous school-based intervention studies (e.g., 
Roberts et al., 2010) were used to estimate the magnitude of these 
interactions, and the degree to which scores on the outcomes 
would cluster within-schools. These estimates were input to 
Murray and Hannahs (1990) formula and then multiplied by the 
design effect — a number that accounts for the reduction in effec- 
tive sample size due to intra-class clustering in the data (Campbell 
et al., 2004). The computation indicated that 12.86 students from 
each of the seven schools in Analysis 1 (N = 196.74) and 15 stu- 
dents from each of the six schools (AT = 216) in Analysis 2, should 
give the GLMM an 80% chance of capturing "small to moderate" 
interaction effects (i.e.,/ = 0.15) at an alpha-level of 0.05. Unlike 
repeated measures ANOVA (or ANCOVA), GLMM does not rely 
on participants providing data at every assessment point; GLMM 
uses all the data present at each assessment point thereby reduc- 
ing the impact of subject attrition on statistical power. Moreover, 
GLMM is robust to unequal group sizes. 

RESULTS 

Two separate analyses were conducted to evaluate the enhanced 
AO-PTS program as fewer control group data were available than 
intervention group data. A primary analysis was conducted on 
seven intervention schools, which consisted of 530 Years 4 and 
5 students, to compare the pre-intervention to post-intervention 
student and parent outcomes (Analysis 1). A secondary analysis 
was conducted comparing three intervention and three control 
schools, consisting of 371 Years 4 and 5 students (Analysis 2). The 
schools were matched in pairs with regards to SES and number of 
Year 4 and 5 students attending the school. 

ATTRITION AND MISSING VALUES 

Data were screened for accuracy. Missing values due to partially 
completed questionnaires were replaced with estimates derived 
from the expectation-maximization algorithm (e.g., Rubin et al., 
2007; Graham, 2009). 



Student attrition 

In Analysis 1, the student cohort comprised 471 students at pre- 
test, of which 458 responded at post-test (a 2.76% attrition rate). 
Significant differences were found for the means of completers 
and non-completers on the mean Total Emotion Attribution 
Score, the mean Total Social Behaviors scores (and the related 
subscales for the emotions Happy and Mad), as well as on the 
Total Social Situations scores (and the related subscale for the 
emotion Scared) (ps < 0.05). With the exception of the Happy 
Social Behaviors subscale, the completers showed higher rates of 
emotion attribution accuracy compared to the non-completers. 
In Analysis 2, the student cohort comprised 353 students at pre- 
test, of which 325 responded at post-test (a 7.93% attrition rate). 
The rate of attrition between the intervention schools (5.55%) 
and control schools (19.61%) was sig nificant, / 2 (1) = 35.07, 
p < 0.001. Significant differences were obtained on the pre-test 
outcomes between those students who were retained and those 
who dropped out, such that the completers had significantly 
higher emotion attribution accuracy on the Total Social Situations 
subscale, and on the Sad Social Behavior subscale. 

Parent attrition 

In Analysis 1, the parent sample at pre-test comprised 320 par- 
ents, of which 191 responded at post-test (a 40.31% attrition 
rate). There were no significant differences on any of the parent 
pre-test outcomes between the completers and non-completers. 
In Analysis 2, the parent sample at pre-test comprised 241 par- 
ents, of which 162 responded at post-test (a 32.78% attrition 
rate). Attrition rates differed significantly between groups — with 
43.56% non-completers for the intervention schools and 35.58% 
non-completers for the control schools, x 2 Q) = 4.34, p = 0.037; 
however, there were no significant differences on any of the parent 
pre-test outcomes. 

STUDENT-REPORTED OUTCOME 

Table S4 presents the pre- and post-test estimated means and 
standard errors on the ACES for the interventions schools in 
Analysis 1, and for the intervention and control schools in 
Analysis 2. In Analysis 1, there were significant effects of time 
on children's Total Emotion Attribution Score, 92 9) = 67.69, 
p < 0.001; Total Social Behavior, _F ( i. 929 ) = 19.96, p < 0.001; 
and Total Social Situations, 929) = 27.90, p < 0.00. 
Significant time effects were also found for accuracy of attributing 
Happy Social Behaviors, _F(! 92 9) = 11.64,p = 0.001; Mad Social 
Behaviors, F(i_ 929) = 26.37, p < 0.001; and Ambiguous Social 
Behaviors, 929) = 340.99, p < 0.001; Mad Social Situations, 
929) = 7.32, p = 0.007; and Scared Social Situations, 
F(i, 929) = 28.06, p < 0.001. No significant time effects were 
found on other Behavior or Social Situation subscales. Table 
S4 indicates that there was also a trend for improvements in 
children's ability to accurately attribute emotions in scales that 
did not reach significance. 

In Analysis 2, there was only one significant interaction effect 
for the Sad Behavioral Situations subscale, F(i i 676) = 17.91, 
p < 0.001; LSD post-hocs indicated that, although significant 
improvements were seen for both the intervention (p = 0.002) 
and control (p < 0.001) school children, this improvement 
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was greater in the control schools — indicating no interven- 
tion effect. There were significant Time effects found on 
the Total Emotional Attribution Scale, Fq : 676) = 58.66, p < 
0.001; Total Social Situations, Fq, 676) = 10.58, p = 0.001; 
and Total Social Behaviors, Fq, 924) = 10.58, p = 0.001. There 
were also significant Time effects found on Sad Social 
Behaviors, Fq, 676) = 71.08, p < 0.001; Mad Social Behaviors, 
F(i, 676) = 9.88, p = 0.002; Scared Social Behaviors, Fq, 6 76) = 
17.78, p < 0.001; and Ambiguous Feelings Social Behaviors, 
Fq, 676) = 759.66, p < 0.001. Similarly, significant Time effects 
were observed for Happy Social Situations, Fq, 676) = 12.95, 
p < 0.001, and Scared Social Situations, Fq, 6 76) = 13.92, 
p < 0.001. These significant Time effects indicated that both the 
intervention and control school children had showed improve- 
ments in their overall ability to accurately attribute emotions 
across social behaviors and social situations (see Table S4). There 
were no other significant times or group effects on the other 
scales. Once again there was a tendency for outcomes to change 
positively over time for both groups. 

Gender differences 

In Analysis 1, there was a significant main effect of gender on 
Emotion Attribution Score, Fq, 395) = 11.71, p = 0.001; such 
that females had higher rates of Total Emotion Attribution com- 
pared to males at both Time 1 and Time 2. There was no sig- 
nificant Gender*Time interaction, Fq, 895) = 41.31, p = 0.125. 
In Analysis 2, there was a significant GenderTime interaction, 
F(i, 661) = 10.67, p = 0.001. LSD contrasts indicated that males 
had a significantly greater pre-post increase [Time 1: M = 18.62 
(SE= 0.31) to Time 2: M= 19.68 {SE = 0.19);p < 0.001], com- 
pared to females [Timel:M= 19.92 (SE = 0.34) to Time 2: M = 
20.60 (SE= 0.28); p < 0.001]. The findings also suggest that 
overall girls across both groups had significantly higher rates of 
overall Emotion Attribution Accuracy. The Gender* Group* Time 
interaction for Total Emotion Attribution Score was not signifi- 
cant, Fq, 661) = 0.44, p = 0.510, indicating no differential impact 
of the intervention depending on gender. 

Grade differences 

In Analysis 1, there was a significant Grade effect, Fq, 92 7) = 9.43, 
p = 0.002, such that Year 5 students had significantly greater 
Emotion Attribution Scores at Time 1 and Time 2, compared to 
Year 4 students. There was no significant Grade*Time interac- 
tion on Emotion Attribution Score, Fq, 927) = 2.12, p = 0.145. In 
Analysis 2, there was a significant Grade* Group* Time interaction 
on Total Emotion Attribution Score, Fq, 372) = 7.96, p = 0.005. 
LSD contrasts indicated that for the Year 4s, there was no pre- 
post change in the intervention group (p = 0.056), although this 
approached significance, but a significant pre-post increase in the 
control group (p < 0.001); for Year 5s, there was a significant 
pre-post reduction in the intervention group (p < 0.001) and an 
equivalent pre-post reduction in the control group (p < 0.001). 
These results indicate that there were no preferential interven- 
tion effects for Year 4s or Year 5s. There was also a significant 
grade effect, Fq, 6 72) = 7.51, p = 0.006, indicating that the Year 
5 children had higher emotion attribution accuracy at both Time 
1 and Time 2, compared to Year 4 children. 



PARENT-REPORTED OUTCOME 

In Analysis 1, there were significant effects of time on the SDQ- 
P Total Difficulties Score, F(i. 510) = 9.27, p = 0.002, and on 
Hyperactivity/Inattention Problems, Fq, 510 ) = 8.95, p = 0.003, 
indicating that parents of students who had received the AO-PTS 
program reported significantly reduced levels of total difficulties 
and hyperactive/inattentive symptoms in their children at post- 
test (see Table S5). In contrast, there were no significant Time 
effects found on Conduct, Peer, and Prosocial subscales (ps > 
0.05). However, Table S5 shows a trend across all five SDQ-P sub- 
scales for improvements in parent-reported scores, with reduced 
levels of Emotion, Conduct, Hyperactivity, and Peer Problems 
and increases in Prosocial Behavior. 

In Analysis 2, there was only one significant interaction 
effect on the Peer subscale of the SDQ-P, Fq, 400) = 5.28, p = 
0.022; however, LSD post-hoc analyses indicated that this effect 
was in an unexpected direction, such that parents of inter- 
vention children reported significant increases in their chil- 
dren's level of Peer Relationship Problems (p = 0.012), whereas 
there was no significant change in control school children (see 
Table S5). Significant Time effects were observed for Total 
Difficulties Scores, Fq, 400) = 17.01; p = 0.042; as well as on 
Emotion, Fq, 40 o) = 7.14, p = 0.008; and Conduct Problems, 
Fq, 400) = 23.01, p < 0.001, such that parent-reported levels 
of problems in these area significantly reduced in both the 
intervention and control school children. In addition, there 
were significant group differences found on Total Difficulties, 
Fq 400) = 23.07, p < 0.001, and Hyperactivity/ Inattention 
scales. Inspection of the means indicate that control school chil- 
dren had higher levels of both Total Difficulties and Hyperactivity 
at both pre- and post-test assessments. No other significant 
group or time effects were observed. Although, the Time effect 
for Hyperactivity/ Inattention Problems approached significance, 
F(i, 400) = 3.70,p = 0.055. 

Gender differences 

In Analysis 1, the Gender* Time interaction on the Total 
Difficulties Score was not significant, Fq, 474) = 0.05, 
p = 0.825. The main effect of gender approached signifi- 
cance, Fq, 474) = 3.69, p = 0.055. In Analysis 2, there was 
a significant Gender* Group* Time interaction on The Total 
Difficulties Scale of the SDQ, Fq, 386 ) = 29.05, p < 0.001. LSD 
comparisons indicated that, for males, there was no pre-post 
change in the intervention group (p = 0.911), but a significant 
pre-post decrease in the control group (p < 0.001). For females, 
there was a significant pre-post reduction in the intervention 
group (p < 0.001) and control group (p = 0.002). These results 
indicated no preferential outcomes due to the intervention. There 
was also a significant main effect of gender, Fq, 386) = 48.12, 
p < 0.001, and time, Fq 38 6) = 14.04, p < 0.001; a comparison 
of the mean Total Difficulties indicated that females across both 
groups had significantly lower Total Difficulties compared to 
males at pre- and post-test. 

Grade difference 

In Analysis 1, there was no significant Grade* Time interaction 
Scores or Grade effect on Total Difficulties (ps > 0.05). In Analysis 
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2, there was a significant Grade* Group* Time interaction on 
Total Difficulties Scores, F(i, 396) = 6.00, p = 0.015. LSD con- 
trasts found that, for the Year 4 students, there was no pre-post 
change in the intervention group (p = 0.742) but a significant 
pre-post decrease in the control group (p = 0.001); for Year 5 
students, there was a significant pre-post reduction in the inter- 
vention group (p < 0.001) but no pre-post change in control 
group (p = 0.089). These results indicated that the Year 5s, but 
not the Year 4s, benefitted from the intervention; although, the 
change in means was small and may not be clinically meaningful 
and the Year 4 control children also improved. 

DISCUSSION 

The aim of the current study was to investigate whether the 
enhanced AO-PTS, a universal schools-based program developed 
for the prevention of anxiety and depression would have a pos- 
itive impact on children's social and emotional competence in 
middle primary school children. The findings from Analysis 1, 
provided some encouraging findings for the efficacy of AO-PTS 
on children's SEL skills and provided partial support for the two 
hypotheses in this current study; however, in Analysis 2, when 
three intervention schools were compared with three-matched 
control schools, there was no preferential outcomes for the inter- 
vention over control schools. In fact, there was a tendency for 
both groups to improve. 

BEHAVIORAL AND EMOTIONAL WELL-BEING 

Partial support was found for the hypothesis that parents will 
report lower levels of internalizing and externalizing symptoms 
and higher levels of total difficulties and prosocial behavior in 
their children at post-test, compared to pre-test levels; how- 
ever, this support was only obtained in the primary anal- 
ysis that comprised of intervention schools — with significant 
reductions in parent-reported overall Total Difficulties and 
Hyperactive/Inattention Problems in their children. There was, 
however, no support with relation to the other subscales of 
the SDQ-P (Emotion, Conduct, Peer Problems, and Prosocial 
Behavior). 

The finding of reduced total difficulties in the primary analy- 
sis of intervention schools is consistent with two studies of Aussie 
Optimism for older children and of the AO-PTS that also found 
reduced total difficulties (Swannell et al., 2009; Roberts et al., 
2010; Rooney et al., 2013). The finding of reduced hyperactivity in 
the primary analysis was consistent with Rooney and colleague's 
findings (Morrison et al., 2013) and findings from the secondary 
analysis which found no significant difference between inter- 
vention and control children on parent reported Hyperactivity, 
Emotion, Conduct or Peer Relation problems were not consis- 
tent with Rooney et al. (2013), Morrison et al. (2013). On the 
other hand, the finding from the secondary analysis indicating an 
increase in peer-related problems was not consistent with Lacey 
et al. (submitted). 

The current study findings of no significant improvements in 
Prosocial Behaviors, Conduct Problems, or Emotion Problems, 
were inconsistent with other studies which found increases in 
prosocial behaviors (Payton et al., 2008; Conduct Problems 
Prevention Research Group, 2010; Durlak et al., 201 1; Jones et al., 



2011; Schonert-Reichl et al., 2012), reduced conduct problems 
(Payton et al, 2008; Swannell et al, 2009; Conduct Problems 
Prevention Research Group, 2010; Durlak et al., 2011; Jones et al., 
2011; Schonert-Reichl et al., 2012), and reduced internalizing 
behaviors (Payton et al., 2008; Durlak et al., 2011). These differ- 
ences in findings may be related to differences in study sample, 
measurement instruments, programs, and raters. 

The unexpected finding that intervention children signifi- 
cantly increased in parent-reported Peer Problems is not con- 
sistent with findings from previous research; for instance, the 
AO-PTS large scales study found significant decrease in peer- 
relation problems (Rooney et al., 2013). Furthermore, this finding 
is in contrast to the enhanced AO-PTS small pilot study which 
demonstrated a preventive effect on the peer problems (Lacey 
et al., submitted) and a Strong Start program piloted on one class 
of students that showed an improvement in Peer Relationships, 
compared to another class who received mathematics class who 
worsened in peer relationship (Caldarella et al., 2009). It is diffi- 
cult to explain why the intervention schools in Analysis 2 would 
worsen in their parent-reported peer problems. Examination 
of the children in the secondary analysis of three intervention 
schools showed that there was a greater proportion (72%) of Year 
4's compared to the control schools (51%). Hence, the increased 
peer problems in the intervention schools may be due to the fact 
that Year 4s had greater level of peer problems than the Year 
5s. Examination of means showed that Year 4 children had con- 
sistently more problems and lower levels of prosocial behavior 
compared to Year 5 s. These differences may indicate developmen- 
tal differences such that the younger group of children are still 
learning and consolidating these skills, whereas the older children 
have had more time to practice. 

Unlike the current study, Swannell et al. (2009) found gen- 
der differences in the 20 week version of Aussie Optimism, such 
that males' prosocial behavior worsened, whereas females had 
significant improvements in their emotion, conduct, and peer 
problems. In this study, however, gender differences were found 
across groups with both intervention and control girls having 
lower levels of total difficulties than both groups of boys. 

EMOTION AND SOCIAL KNOWLEDGE 

Partial support was found for the hypothesis that intervention 
children will report higher levels of emotion attribution accuracy 
in Analysis 1, with significant increases on the overall emotion 
attribution accuracy score; on the total emotional attribution of 
social behaviors and the related subscales happy, mad, and no 
feelings; and on the total emotional attribution of social situ- 
ations and the related subscales mad and scared. However, no 
preferential improvements were found in favor of the interven- 
tion children when compared to control school children and thus 
the intervention effect may be due to extraneous factors. 

When considering the primary analysis of intervention schools 
only, the findings of improvements in emotion knowledge are 
consistent with those found in one of the PATH's studies 
(Domitrovich et al., 2007) and in an emotion-based prevention 
program of 2- to 5-year old children who were reported to have 
greater increases in emotion knowledge and social competence 
(Izard et al., 2008). However, the above positive findings did not 
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remain in Analysis 2; and the lack of improvements in children's 
empathy or perspective taking are consistent with findings in the 
"roots of empathy" study (Schonert-Reichl et al., 2012). 

In the current study gender and age differences were found 
in both groups. Intervention and control girls had significantly 
higher emotional attribution accuracy than boys at pre- and post- 
test. Year 5 students in both groups, had higher total emotion 
attribution accuracy, compared to Year 4 students. 

There are several potential explanations for the age differ- 
ence findings. According to Piaget's theory, the middle childhood 
period is a time when children are moving into the concrete 
operational stage, between the ages of 7 and 12 years; this is a 
period of time when children start to reason more logically about 
concrete objects and events (Piaget, 1952). Thus, perhaps the 
younger students in this study were less well-developed than their 
Year 5 counterparts. In addition, research on brain development 
has indicated that important changes take place during middle 
childhood, with improvements in prefrontal cortex functioning 
related with planning, decision-making, emotion regulation, and 
abstract thought (Steinberg, 2005). It is therefore possible that 
the differential effects found between the Year 4 and 5 children 
are due to the elder children being more advanced in their brain 
development and cognitive ability, which may translate to better 
SEL capability, such as regulating themselves, planning behav- 
ior and problem solving in social situations and having better 
impulse control. In addition, perspective taking skills are also 
emerging in middle childhood and children during this period are 
becoming more self-aware, reflective, and less egocentric (Selman, 
1980). Emotion knowledge has been shown to be important in 
children's social and affective development and essential for com- 
petence in social situations (Halberstadt et al., 200 1; Buckley et al., 
2003). Given the importance of emotion knowledge, other mod- 
ifications to AO-PTS program could include an increased focus 
on perspective taking and the greater emphasis on recognition of 
emotions in self and others. 

EXPLANATIONS FOR NON-SIGNIFICANT FINDINGS 

There are several possible explanations as to why there were 
no significant findings for an intervention effect when interven- 
tion schools were compared to control schools. First, assessment 
immediately following intervention at post-test may be too early 
to detect changes in children's SEL skills, as it may take time 
for children to practice these skills and integrate them into 
their behavioral repertoire before changes will be noticeable on 
outcome measures. For instance, other studies have found no 
intervention effects immediately at post-test, but detect interven- 
tion effects at follow-up (Dadds et al, 1997; Quayle et al, 2001; 
Roberts et al., 2003). For example, this delayed intervention effect 
was found for Prosocial and Total Difficulties on the SDQ at 12 
month follow-up, but not immediately post-intervention, in an 
evaluation of Zippy s Friends in a large scale RCT of 6- to 8-year 
old primary school children in Ireland (Clarke, 2011). The larger 
AO-PTS study of 20 schools, from which this data were taken, will 
be collecting data at 6- and 18-months follow-up period and will 
thus be able to assess for delayed intervention effects. However, 
the intervention children would need to show greater improve- 
ment in order to rule out maturation effects. Second, control 



schools may have run other SEL-based programs or have done so 
in the recent past; such programs would account for the equiva- 
lent improvement in control school children emotion knowledge 
and problems. Future research should attempt to collect informa- 
tion on all current or previous SEL programs in schools. Third, 
as part of the larger ongoing AO-PTS research, which investigates 
the effects of the program on anxiety and depression, a report is 
provided to parents on any children who are assessed as "at risk." 
This may have impeded the demonstration of efficacy of the inter- 
vention program as it is possible that parents of control group 
children identified as "at risk" may have sought professional help 
to assist their children to overcome their problems. 

A further possible explanation for the lack of intervention 
effects in comparison to control schools maybe the relatively high 
baseline levels of emotional knowledge and low levels of emo- 
tional and behavioral problems at baseline in this sample, which 
consisted of children from private schools, when compared to 
other Aussie Optimism studies (Swannell et al., 2009; Rooney 
et al, 2013). Previous research has shown that children with 
higher levels of problems at baseline can benefit more from the 
intervention (e.g., Swannell et al., 2009). It could be the case that 
the children in this study were too high functioning to see changes 
in their scores as there was a ceiling effect in place. It would have 
been useful to have included an analysis of children at different 
levels of problems to see if ( 1 ) there were any treatment effects 
(moved into normal range from clinical range) or (2) any pre- 
ventative effects (no increased levels of problems from normal to 
clinical range). This type of analysis would have provided a more 
nuanced view on which type of cases the intervention is effec- 
tive for, which may be lost in a generalized average of pre- and 
post-test scores across all children in a school. 

Another issue which may have hampered efforts to establish 
whether or not AO-PTS has positive impacts on children's SEL 
skills is to do with methods of measurements. First, the ACES 
in this study had inadequate reliability and was unstable across 
time — this may have reduced the power of the analyses to detect 
differences between groups. In addition, the ACES is a measure 
of only one aspect of SEL, namely social awareness. However, 
another way of assessing effectiveness of AO-PTS on SEL may 
have been to investigate whether or not children acquired the SEL 
skills in the program. For instance, other studies have included 
a SEL knowledge acquisition assessment (e.g., Strong Start pro- 
gram, Whitcomb and Merrell, 2012). A further change to the way 
SEL was measured could be to include different SEL measure- 
ment tools that tap into more of the five core elements of SEL. 
There are several recent resources that have listed and summa- 
rized the psychometric properties and applications of different 
SEL measures in children (e.g., Denham et al., 2010; Schonert- 
Reichl et al., 2010; Haggerty et al., 201 1). A further measurement 
issues is who is rating the SEL skills. Evidence suggests that dif- 
ferent raters can provide different ratings. For instance, one study 
found no change in skills or levels of problems when parents pro- 
vided ratings of their children, however, significant improvements 
in children's outcomes were seen when teachers rated the children 
(Dadds and Roth, 2008). In the current study, levels of problems 
and prosocial behavior were rated by parents. Parents may have 
less opportunity to view changes in their children's SEL skills as 
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there is potentially less variety of social interactions in the home 
environment. Thus, future research should also collect teacher's 
ratings to reflect changes that may be evident in the school context 
and provide the opportunity to triangulate outcome data. 

The non-significant findings in this study may also have been 
a result of low implementation quality. Although the program 
offered five coaching sessions, only small numbers of teachers 
made use of this opportunity. Unfortunately, these teachers were 
only coached in one session due to teachers' time constraints and 
schedule. One limitation of this study was the lack of implemen- 
tation fidelity assessment; thus, it is not known to what extent 
the intervention was implemented as planned. Fidelity checks 
are essential as programs with no implementation problems are 
associated with better outcomes (Durlak et al., 2011). In future 
studies of AO-PTS, when recruiting and training school teach- 
ers, the importance of implementation fidelity components of 
the study need to be stressed. One possible strategy to boost the 
implementation quality is to re-design the teacher's logbook to 
make it shorter, concise and not to be perceived as extra burden to 
teachers' current job capacity. It has been found that the teaching 
context, workload and satisfaction affect teachers' performance in 
class (Smith and Bourke, 1992). 

Another possible limitation of this study is related to teachers' 
attitudes and beliefs about the effectiveness of the interven- 
tion program. Studies have shown that teachers' behavior and 
teaching practices can be strongly predicted by their attitudes 
and beliefs (Kagan, 1992; Rimm-Kaufman and Sawyer, 2004). 
Furthermore, it has been found that teachers' beliefs about social 
and emotional regulation anticipate teachers' self-awareness and 
recognition regarding skills practices (Van Ewijk and Van Der 
Werf, 2012). Students will sustain the new skills learnt and imple- 
ment these in and outside the classroom if they are taught by 
teachers who have robust beliefs toward the effectiveness of the 
intervention program and who have demonstrated high levels of 
self-efficacy and positive attitudes (Rosen et al., 2009). The Year 
4 and 5 teachers involved in the current study were asked to con- 
duct the program by the school principals and attended a 1 day 
workshop to equip and prepare them on the theories and activi- 
ties required to run the program in class. Some teachers were not 
keen to participate and were reluctant about being observed when 
implementing the program and this may have influenced the 
effectiveness of study outcomes. A baseline assessment is needed 
to assess teachers' overall view on the value of mental health pre- 
vention programs like the AO-PTS to determine any factors that 
may promote or hinder program efficacy. However, the drawback 
in assessing teachers' beliefs and attitudes in the current study is 
that it may have been perceived as additional workload for the 
participating teachers as this kind of assessment usually takes the 
form of a self-report (Van Ewijk and Van Der Werf, 2012). A fur- 
ther issue with self-report of this kind is the potential for teacher 
bias in responding in favor of the expected results in order of 
promote mental health programs in the school context. 

FUTURE DIRECTIONS 

Many future directions for AO-PTS and other universal SEL 
program research have already been discussed. Another future 
direction for SEL program research is to link the acquisition and 



presence of specific SEL core elements with impacts on specific 
types of outcomes and for which types of children and cases. This 
type of information will guide the development and modification 
of SEL universal program to best target and direct resources into 
those skills that are linked with positive outcomes. Also, poten- 
tial age and gender differences highlight the need to include these 
factors as part of analyses. 

CONCLUSION 

This study investigated the efficacy of a school-based universal 
program that is delivered by teachers as part of regular class cur- 
ricula on middle primary school children's SEL skills. This study 
highlights how different results can be obtained with or with- 
out a comparison group, and call into question findings which 
do not include control groups. At this point in time, this study 
showed no strong evidence to support the efficacy of AO-PTS on 
improving SEL skills of Year 4 and 5 children. Methodological 
issues will need to be addressed in future research. AO-PTS has 
established evidence for its effectiveness (i) on prevention and 
treatment of depression symptoms and disorders, (ii) to increase 
positive attribution, (iii) to reduce the children's total difficulties 
problems, and (iv) to reduce hyperactive behaviors. However, its 
effectiveness of SEL skills remains to be established and needs to 
be investigated further. 

SUPPLEMENTARY MATERIAL 

The Supplementary Material for this article can be found 
online at: http://www.frontiersin.org/journal/10.3389/fpsyg. 
2014.00909/abstract 
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